Thesis Outline
- replace I&I with I&I (water intrusion)
1. Chapter 1: Machine learning applications in Sewer Systems
1.1 Introduction:
1.1.1. Sewer Layout and issues in Sewer, the need for asset management
Much of the water infrastructure in the United States, Western Europe, and many other places is aging and in serious need of replacement or upgrading, especially to address the effects of a changing climate and new generation of man-made contaminants. [reference]](https://www.pewtrusts.org/en/trend/archive/spring-2019/how-development-of-americas-water-infrastructure-has-lurched-through-history)
1.1.1.1 Sewer Layout
Three main types of sewers:
Sanitary sewers: collect wastewater from household and businesses and transport it to a wastewater treatment plant (WWTP) before discharging to open water
Storm sewers: collect surface runoff due to rain fall or snowmelt from sidewalks and road and transport it to a nearby surface water with or without treating it
Combined sewers: collect and transport both surface runoff and wastewater to WWTP before discharging to open water. Combined sewer was a practice in the 19^th^. Therefore, many older sewer systems around the world consist of combined sewers.
Type of conveyance system:
Gravity: pipes are installed on grade and might require lift station
Pressure: no grading on the pipe and generally smaller pipes when comparing to gravity pipes. Require installing grinder pumps at each unit.
Vacuum: require a centralized pumping station and collection pits
1.1.1.2. Issues
CSO and SSO:
Most older sewage systems in North America are combined sewers. During extreme precipitation events or snowmelt events, wastewater has to diverge to the outflow and directly release to open water without treating because WWTP is at capacity
Infiltration and Inflow also adds more burden to combined sewer system. I&I flows happen there is a crack inside the pipe or manhole causing groundwater or runoff entering the sewer system
Problem with CSO and SSO:
- Affect water quality of the surface water that the discharge is released into
1.1.2. What is the state of usage of machine learning in sewer systems in general
1.1.2.1. Modelling of Sewer Network Flow
(Marinaki-Papageorgiou2005_Book_OptimalReal-timeControlOfSewer)
The first step in the study and control of a process is the development of a mathematical model of the process behaviour. The mathematical model includes a set of equations that describe, with more or less accuracy, the behaviour of the process, and through this model (using appropriate input data) the time evolution of internal quantities of the process may be calculated. The mathematical model of the process may be derived via:
The deductive way, by using known laws of physics that describe the relevant aspects of the studied process, as for example, the continuity equation. In the case of a sewer network, the model that is deduced in this way is referred to as the hydraulic or hydrodynamic model (Béron and Richard, 1982).
The inductive way, by using experimental results (input-output values) that indirectly characterize the behaviour of the process.
A combination of both previously mentioned approaches, using physical laws and experimental results. A mathematical structure derived from physical laws may be fitted to the data, as it is the case for the sewer network flow when hydrological models (Béron and Richard, 1982) are developed
1.1.2.2. In planning
- Traditional way:
- Sewer is designed based on projected flow (usually design to take into account for population growth)
1.1.2.3. In operation
(Marinaki-Papageorgiou2005_Book_OptimalReal-timeControlOfSewer)
Typically, two models are developed in control applications.
The first is a sufficiently accurate simulation model that describes the behaviour of the process with high realism in order to be usedfor testing the performance of the control methods.
The second model has lower accuracy and complexity and is used for the design of the controller, leading to accordingly limited design complexity and moderate computational effort.
Influent Data to predict Effluent data
- Data-based approach (to develop hydrologic modelling) can be used to predict the system overflow behaviour during rain events more realistically than in conventional sewer system modelling.
1.1.2.4. In maintenance
1.2. Literature review
What are the gaps in the literature review, and why are these gaps present?
- The condition score is arbitrary. It is better to create a model that can predict the likelihood and defect it is most likely to occur. And pipes can be scheduled for replacement based on their defects.
What is the research question that we are going to address? Why is what you are doing important and why has it not been done before? How am I going to use AI in collection system asset management?
The benefit of the AI model (keep coming up with the benefit of AI)
Require little to no domain knowledge.
They use data that are measurable (such as flow). Unlike the physical model that would require imperial parameters (such as roughness coefficient) that cannot easily be measured.
Learn by looking at the pattern of input and output data.
The model can be trained using both simulation and field data.
The disadvantage of AI
Require large data to train a model.
Because the model is learned via input data, the model's performance depends on the quality of the input data.
Data-driven vs Physical model
Data-driven approaches use information from previously collected data (training data) to identify the characteristics of the currently measured pressure, temperature, or production rate and to predict the future trend. Physics-based approaches assume that a physical model describing the behavior behind these measurements is available and somehow sufficiently accurate and self-contained to predict future behavior. reference
Roadmap of my thesis
2. Chapter 2: Collection Systems Asset Management
2.1. Sewer asset management
Tscheikner-Gratl et al. (2019) present the state of the art in sewer asset management. The paper emphasizes the essence of knowing the current condition of the sewer to develop efficient sewer rehabilitation strategies. The paper begins with the topic of inspection using closed circuit television (CCTV) and alternative inspection physical techniques, such as laser scanning and distributed temperature sensing (DTS), that can provide more information on the type and severity of defects and not just visual inspection. One of the issues with these inspection methods is that it is labour intensive and time-consuming. The interpretation of data is prone to human error. The current need in data management is to have a collection of data that is consistent and organized to the same standard over time.
In addition, the paper mentions that using sewer deterioration modelling can be used to estimate the current sewer condition of uninspected sewers and forecasting the future degradation of the network for long-term planning. There are three types of models: deterministic, statistical, and machine learning models. Survival analysis and Markov models are found to be effective approaches at the network level while machine learning and statistical regression are found to be effective approaches at pipe level. Lastly, a multi-criteria decision analysis (MCDA) or input for cost-benefit analyses (CBA) can be used to aid in decision-making. Both direct and indirect costs and benefits would be considered when failures of the sewer network occur. (Tscheikner-Gratl et al., 2019)
3. Chapter 3: Sewer pipe assessment methods and LoF calculation
3.1. Different Sewer pipe condition inspection methods
Figure 1: Overview of main sewer inspection technologies. In-line technologies are applied from inside the sewer. On-line technologies are also applied from inside the sewer but require contact with the sewer material. (Kley et al., 2013)
3.2. Different Pipe condition scoring standards
Including the evolution of different coding standards and list of countries that use those standard.
Fotomanual (Danish sewer inspection standard) (used by SewerML data)
WRc MSCC Manual of Sewer Condition Classification (UK) (used by DNV until 2018) (I have to double check)
NASSCO PACP (used by DNV 2018-present)
Structural defect coding
Operation and maintenance coding
Construction features coding
Miscellaneous feature coding
3.3. Different machine-learning image-based models in Sewer
| Models | Algorithms | References |
|---|---|---|
| Detect three types of defects (root intrusion, deposits, and cracks) | CNN | Kumar et al. 2018 |
| Defect classification with 47000 images of defect longitude, debris silty, joint faulty, join open, lateral protruding, and surface damage | CNN | Hassan et al. 2019 |
| Classify and detect the sewer defects | Deep object detection and recognition algorithms (SSD, YOLO, Faster R-CNN) | Kumar et al. 2020 |
| Classify and detect the sewer defects | Semantic segmentation for pixel labelling | Wang and Cheng 2019 |
| Multi-label sewer defect classification (test six methods from the sewer defect classification domain and six methods from the general multi-label classification domain) | Two-stage approach: 1) small CNN for an initial binary defect classification; 2) larger CNN, TResNet-L for multi-label classification step | Haurum and Moeslund 2021 |
| Automated anomaly detection and localization in sewer CCTV inspection videos | Proportional Data Modeling and Deep learning-based text recognition | Moradi et al. 2020 |
| Pipe Sleuth (13 PACP classes from 26,600 images) | Faster R-CNN ResNet101 | Singh et al. 2019 |
| Water-level boundary line detection | Deep learning (DeepLab v3) and image processing method | Ji et al. 2020 |
| Water-level estimation | CNN | Haurum et al. 2020 |
| Multi-Task classification of sewer pipe defects and properties | Cross-Task Graph Neural Network Decoder (CT-GNN) | Haurum et al. 2022 |
- Using deep learning-based methods for sewer defect classification, detection, segmentation, and spatiotemporal-based analysis
- Two-step approach model: 1) binary defect/non-defect classification and 2) defect classifier.
- Water level classification in the pipe model
- Multi-Task classification of sewer pipe defects and properties: Identify water level, defect, pipe material, and pipe shape needed to determine the deterioration score (Multi-Task classification of sewer pipe defect and properties using a cross task graph neural network decoder)
- Moradi et al. (2020): The developed algorithms employ three-dimensional (3D) Scale Invariant Feature Transform (SIFT) to extract spatiotemporal features in sewer CCTV videos. Anomaly detection is performed using a one-class support vector machine (OC-SVM) trained by frames without defects to model states considered normal and to classify outliers to this model as anomalous frames. Then, the identified anomalous frames are located by recognizing included text information in them using an end-to-end text recognition approach. The proposed localization approach is divided into two main steps: text detection using maximally stable extremal regions (MSER) algorithm and text recognition using a deep convolutional neural network (CNN).
References
Hassan, S. I., L. M. Dang, I. Mehmood, S. Im, C. Choi, J. Kang, Y. Park, and H. Moon. 2019. "Underground sewer pipe condition assessment based on convolutional neural networks." Autom. Constr. 106 (Oct):102849. https://doi.org/10.1016/j.autcon.2019.102849.
Haurum, J.B. and Moeslund, T. B. 2021. "Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark."
Haurum, J.B., Bahnsen, C. H., Pedersen, M., & Moeslund, T. B. 2020. "Water Level Estimation in Sewer Pipes Using Deep Convolutional Neural Networks."
Haurum, J.B., Madadi, M., Escalera, S., & Moeslund, T. B. 2022. "Multi-Task Classification of Sewer Pipe Defects and Properties using a Cross-Task Graph Neural Network Decoder."
Ji, Hyon Wook, Sung Soo Yoo, Bong-Jae Lee, Dan Daehyun Koo, and Jeong-Hee Kang. 2020. \"Measurement of Wastewater Discharge in Sewer Pipes Using Image Analysis\" Water 12, no. 6: 1771. https://doi.org/10.3390/w12061771
Kumar, S. S., D. M. Abraham, M. R. Jahanshahi, T. Iseley, and J. Starr.
- "Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks." Autom. Constr. 91 (Jul): 273--283. https://doi.org/10.1016/j.autcon.2018.03.028.
Kumar, S. S., W. Mingzhu, D. M. Abraham, M. R. Jahanshahi, I. Tom, and J. C. Cheng. 2020. "Deep learning--based automated detection of sewer defects in CCTV videos" J. Comput. Civ. Eng. 34 (1): 04019047. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000866.
Moradi, S., Zayed, T., Nasiri, F., & Golkhoo, F. 2020. "Automated Anomaly Detection and Localization in Sewer Inspection Videos Using Proportional Data Modeling and Deep Learning-Based Text Recognition."
Singh, K., Varadarajan, S., Guruvayurappan, S., & Anand, A. 2019. "Pipe Sleuth with Optimized Inference on Intel Processors."
Wang, M., and J. C. P. Cheng. 2019. "A unified convolutional neural network integrated with conditional random field for pipe defect segmentation." Comput. -Aided Civ. Infrastruct. Eng. 35 (2): 162--177. https://doi.org/10.1111/mice.12481.
3.4. Sewer pipe condition model
Usually, not all pipes are inspected. Therefore, pipe conditions of uninspected can be estimated via sewer deterioration models. Kley & Caradot (2013) listed the two purposes of deterioration models: i) to simulate the current condition of non-inspected using the condition data of a part of sewer network that are available and ii) to forecast the future degradation of the network. Figure 3 shows how visual inspection from CCTV can be used to model the deterioration model. Figure 4 shows the three basis groups of existing sewer deterioration model. The models then can be further divided into pipe group (network) and pipe level models.
Cohort survival and Markov models are found to be useful approaches for the pipe group level degradation modelling (Kley & Caradot, 2013). Regression models are better suited for identifying the basic relationships between the variables that contribute to the condition of the pipe while machine learning based models have better prediction capabilities (Baah et al., 2015). However, the quality of prediction of these models depends on the extensive dataset of inspection data.
Caradot et al. (2018) investigated the performance of a statistical deterioration model to simulate the current condition of the pipe and predict the future deterioration of a network by comparing GompitZ, an advance approach, to simple model approach. They found the simple model is better than GompitZ in estimating the current condition. However, GompitZ performed better in predicting future conditions by successfully simulating the condition of old sewers.
Hernandez et al. (2018) did a study to compare the prediction results of two models, Logistic Regression (LR) and Random Forest (RF) methods. The results show that RF method has a higher capacity of prediction than LR. On the other hand, Robeles-Velasco et al. (2021) show the combination of LR and genetic algorithm (GA) work well to predict pipe failure. The benefits of different prediction methods can vary based on different management objectives. The goal is to create a predictive system that is autonomous and independent to support the decisions about pipe replacement plans of companies.
3.4.1. Determine the correlation between environment condition and pipe condition
- use condition data and gis
3.4.2. Predict the likelihood of defects occurring in the uninspected pipe using feature engineering
- should include references related to this
3.4.2.1 Factors influencing sewer pipeline deterioration (NASSCO)
Structural:
Soil quality
Position of groundwater table
Loads
Original pipe strength and its loss over time
Alignment and sags
Mortar loss/Bricks missing in walls of pipe
Maintenance
Cleaning methods
Roots
Fats, Oils and Grease (FOG)
Obstructions/blockages
Improper Pipe repairs
Poor access to manholes for maintenance
Hydrogen Sulfide (H2S) attack or other chemical attack
Construction/Design
Surcharging
Quality of construction
Defective lateral connection methods and other defective junctions
4. Chapter 4: CoF calculation
5. Chapter 5: Risk calculation
6. Chapter 6: Replacement/Rehab scheduling
Random inspection of sewers in order to assess the condition is extremely expensive. Thus, it is important to prioritize inspections to those sewers that are more vulnerable to deterioration phenomena and have a higher risk of collapse. There is, therefore, a need to develop a preinspection tool that enables the decision maker to highlight the critical sewer spots for further detailed investigation. reference
7. Chapter 7: Climate change impacts on LoF and CoF
8. Research experiment
8.1. Dataset
Challenges: data processing
The behaviour of each inspector is different.
The annotation made in the video and the access database can be different.
Inspections were done by different contractors. Therefore, the inspection software can be different, making it harder to automate the labelling process.
The defect coding standard evolved over time.
In PACP, there are around 200 defect codes. However, only certain number of defects are used.
8.2. Algorithm
fastai, a deep learning library that uses PyTorch as support. Fastai uses pre-trained models such as ResNet50 model that has pre-stored million images which help the model to be trained at a faster rate.
Image Classification: It is said and proved by many Data Scientists in their research over the past years, that Convolution Neural Network (CNN) is the best Deep Neural Network model for image classification or object detection. And there are many reasons for why it is best, as said by an author in his research, in CNNs, the weights of the Convolution layer being used for feature extraction as well as fully connected layer being used for classification which are determined during training process (Samer Hijazi, 2015). He even mentioned that, more improved network layer of CNNs even lead to memory saving and computation complexity. reference
8.3. Models
Sewer defect classification model
Water level classification model
8.4. Ideas
Current sewer defect classification models are utility-specific and defect code specific. No paper has addressed the performance of a model trained on a specific defect code on data from different defect codes when applying transfer learning.
We also want to explore the idea of a continuous learning model. We want to compare the performance of the model that is initially trained by a small batch of data and continuously fine-tuning as new data comes in with a model that is trained by all available data at once.
The defect coding standard evolves over time. The label dataset evolves over time. How should the model be maintained and modified to accept the new standard.
How should the model be improved when the new data comes in
Predict hierarchical labelling